Yelp Restaurant Photo Classification

Everyday, millions of users upload numerous photos of different businesses to Yelp. Though most of them are tagged with simple information such as business names, the deep-level information typically are not revealed until looked by humans. Such deep-level information contains things like whether the restaurant is good for lunch or dinner, whether it has outdoor seating, etc. This information can be crucial in helping potential customers to determine whether its the ideal restaurant they want to dine in. Hand labeling these restaurants can be not only costly but also tedious since photos are uploaded almost every second to Yelp. In this project, we used machine learning techniques, specifically image classification methods, to identify the context and rich information of different photos on Yelp. We used three different models to train and test the given dataset. The dataset is provided by Yelp on Kaggle.com. In the training set, 234842 arbitrary-sized images submitted by Yelp users are correlated to one of 2000 Yelp businesses. The number of corresponding photos for each business ranges from 2 to 2974, with mean of 117. The image count distribution is shown by Fig1. Every business is tagged with multiple descriptive labels in 9 aspects. The labels includes: 0: good for lunch 1: good for dinner 2: takes reservations 3: outdoor seating 4: restaurant is expensive 5: has alcohol 6: has table service 7: ambience is classy 8: good for kids As an example, Fig2 illustrates the structure of the dataset. Our goal is to predict the labels of a business in these 9 aspects given its photos. A notable challenge is that the labels are not directly correlated to each image. Individual photos can possibly be indicative in one aspect, yet it is tagged with all the labels of the business. As a result, the dataset might exhibit high noise level. The fraction of each label across the training set is shown in Fig3. Though assumed independent, some label pairs exhibit high covariance, such as ”has alcohol” and ”has table service” (Fig4).